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[57] ABSTRACT 

The adaptive computing system disclosed herein em- 
ploys a data structure involving a multiplicity of classifi- 
ers, each of which includes a match string of characters 
which the system attempts to match up with messages 
generated either by the environment or by other classi- 
fiers. Associated with each match string is a response 
message definer or action part which defines the re- 
sponse to be given when a match is obtained. To facili- 
tate matching in a noisy or changing environment, there 
is associated with each character in the match string a 
weighting value and the degree of match is judged by 
means of a score value which is a function of the 
weighting values of the characters which match be- 
tween the match string and the message. 
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ABSTRACT : 

The adaptive computing system disclosed herein employs a data structure 
involving a multiplicity of classifiers, each of which includes a match string 
of characters which the system attempts to match up with messages generated 
either by the environment or by other classifiers. Associated with each match 
string is a response message definer or action part which defines the response 
to be given when a match is obtained. To facilitate matching in a noisy or 
changing environment, there is associated with each character in the match 
string a weighting value and the degree of match is judged by means of a score 
value which is a function of the weighting values of the characters which match 
between the match string and the message. 
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Brief Summary Text - BSTX (13) : 

The match string for each classifier is compared to each input signal 
message and a corresponding score value is generated which is a function of the 
weighting values of the matching bits. The score values generated by the 
different classifiers are compared and respective response messages are output 
for classifiers having the better scores . The strength value for each 
classifier whose output message was appropriate is modified as a function of 
the respective reinforcement signal. Periodically, genetic operations are 
applied to generate new classifiers based upon the modification of selected 
ones of the match strings and selected sets of weighting values, the selection 
process including a randomizing component and a component based on the 
respective strength values. 

Detailed Description Text - DETX (9) : 

In accordance with the present invention, genetic operations are performed 
on the weight values associated with each of the bits in the match string as 
well as upon the match string itself. The process is represented in FIG. 1 by 
the block designated by reference character 29. The genetic operations 
performed on the weights may be performed in conjunction with those on the 
match string or may be performed independently thereof. FIG. 3 represents the 
application of a powerful crossover genetic operator applied in coordinated 
fashion to both the match string and the associated weight values. In this 
example, parent 1 comprises the match string 010101 and parent 2 comprises a 
match string which is all zeroes. A crossover point is determined at random. 
In the example illustrated in FIG. 3, the crossover point is assumed to be 
between the second and third bit positions. It should be understood that 
multiple crossover points, also chosen at random, may be usefully implemented 
particularly with longer match strings. With reference to FIG. 2, the 
combining with crossover yields two children of the parents, one with the match 
string 010000 and the other with the match string 000101. In each case in the 
illustrated example, the weight value associated with each bit of the child 
string is taken from the same parent which produced the match bit. An 
alternative, however, would be to average the weights from the two parents for 
each bit position. As is understood by those skilled in the art, this sort of 
crossover genetic operation provides a chance for the generation of new 
classifiers which combine useful aspects of two different parent classifiers. 
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As indicated previously, the selection of parent's is performed as a stochastic 
process with the likelihood of selection being based on the relative strength 
of the various classifiers. On the other hand, since the stochastic process 
includes a random component, a healthy diversity in the overall population is 
maintained. 

Claims Text - CLTX (9) : 

comparing the score values generated by the different classifiers ; 

Claims-Text - CLTX (24): 

comparing the score values generated by the different classifiers ; 
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[57] ABSTRACT 

A system for separating and identifying classes of pat- 
terns or events which are not necessarily linearly sepa- 
rable. During a training process of the system, new 
prototypes are created when prior stored prototypes 
fail to properly classify an input pattern, and previously 
stored prototypes are modified when an input pattern 
falls within the sphere of influence of a prototype asso- 
ciated with a different class than the input pattern. 
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modified when an input pattern falls within the sphere of influence of a 
prototype associated with a different class than the input pattern. 
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Brief Summary Text - BSTX (25) : 

Similarly, although only a single pattern feature encoding device may be 
provided, the provision of a multiplicity of pattern classifiers, looking at a 
different aspect of the encoding device output and each trained in a slightly 
different way, makes it possible to separate the pattern into one of a large 
number of possible classes. It will be appreciated that when a very large 
number of possible classes are involved, the pattern classifier is able to 
distinguish between subtle differences in the patterns which it "sees". 

Claims Text - CLTX (68) : 

52. The pattern classification method defined in claim 51, wherein said 
label of said given prototype is converted from a first phase type to a second 
phase type if said scalar distance thereof is reduced below a prescribed 
minimum value . 
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ABSTRACT 



A classification method and apparatus for classifying an 
input into one of a plurality of possible outputs. Infor- 
mation representative of the input is compared to a 
neuron, where* the neuron comprises a boundary de- 
fined by two or more neuron axes, of different length. 
One of the possible outputs is then selected as corre- . 
spending to the input in accordance with that compari- 
son. The invention is also a training method and appara- 
tus for creating a new neuron or adjusting an existing 
neuron. A feature vector representative of a training 
input is generated, where the training input corresponds 
to one of a plurality of possible outputs. If no existing 
neuron corresponding to the training input encompasses 
the feature vector, then a new neuron is created, where 
the new neuron comprises a boundary defined by two 
or more neuron axes of different length. If the neuron 
encompasses the feature vector and if the neuron, does 
not correspond to the training input, then the neuron is 
adjusted spatially, where the adjusted neuron comprises 
a boundary defined by two or more adjusted neuron 
axes of different length. 
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ABSTRACT: ' 

A classification method and apparatus for classifying an input into one of a 
plurality of possible outputs. Information representative of the input is 
compared to a neuron, where the neuron comprises a boundary defined by two or 
more neuron axes of different length. One of the possible outputs is then 
selected as corresponding to the input in accordance with that comparison. The 
invention is also a training method and apparatus for creating a new neuron or 
adjusting an existing neuron. A feature vector representative of a training 
input is generated, where the training input corresponds to one of a plurality 
of possible outputs. If no existing neuron corresponding to the training input 
encompasses the feature vector, then a new neuron is created, where the new 
neuron comprises a boundary defined by two or more neuron axes of different 
length. If the neuron encompasses the feature vector and if the neuron does 
not correspond to the training input, then the neuron is adjusted spatially, 
where the adjusted neuron comprises a boundary defined by two or more adjusted 
neuron axes of different length. 
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Detailed Description Text - DETX (87) : 

In one preferred embodiment, each cluster classifier may implement a 
classification algorithm unique to that character cluster, or shared by only a 
subset of the total number of character clusters. Each cluster classifier may 
therefore employ neurons that exist in a feature space unique to that character 
cluster. For example, training for the "P", "R" , "B" cluster may employ a 
particular set of Grid features, while training for the "0", "C", "D" , "U" , "Q" 
cluster may employ a different set of Hadamard features. In that case, 
different training procedures are performed for each different cluster 
classifier, where only inputs corresponding to those characters of the 
associated cluster are used for each different training procedure. 

Detailed Description Text - DETX (93) : 

In an alternative embodiment, two or more different classifiers may be 
arranged in parallel. In such case, a voting scheme may be employed to select 
the appropriate output by comparing the outputs of each different classifier . 

Claims Text - CLTX (21) : 

10. The classification method of claim 1, wherein said first number is 
equal to the number of said selected neurons that are associated with said 
first possible output, and said second number that is equal to the number of 
said selected neurons that are associated with said second possible output. 



KWIC 



5/10/05, EAST Version: 2.0.1.4 



United States Patent m 

Engel et al. 



■hiiihiiiimh 



USO05761383A 
Patent Number: 
[45] Date of Patent: 



5,761,383 
Jun. 2, 1998 



[54] ADAPTIVE FILTERING NEURAL NETWORK 
CLASSIFIER 

[75] Inventors: Stephen J. Engel. E. Northport; Dennis 
Buckland. Bethpage, both of N.Y. 

[73] Assignee: Northrop Grumman Corporation. Los 
Angeles, Calif. 

[21] Appi. No.: 867,719 
[22] Filed: Jun. 3, 1997 

Related US. Application Data 

[63] Continuation of Ser. No. 430,529, Apr. 27, 1995, abandoned. 

[51] Int Q. 6 ~. G06F 15/18 

[52] US- a 395/21; 395/22; 395/23 

[58] Field of Search 395/21, 22. 23; 

382/190 

[56] References Cited 

U.S. PATENT DOCUMENTS 



5,003,490 
5,092343 
5,133,021 
5,165,069 
5,263,097 
5386,689 
5,402320 
5,467,428 
5,479372 
5333383 
5379,232 
5386,220 
5,649,065 
5,680,627 



3/1991 
3/1992 
7/1992 
11/1992 
11/1993 
2/1995 
3/1995 
11/1995 
12/1995 
7/1996 
11/1996 
12/1996 
7/1997 
10/1997 



Castelaz et al. 

Spitzer et al 

Carpenter et al. 

Vitt et al 

Katz et al 

Bozich et al 

Schmitta , 

Ulug 

Marcantonio ..... 

Greene et al 

Tong et al 

Spiker 

Lo et al 

Anglea et al 



..... 395/513 
..... 128/733 

382/15 

358/335 

..... 382/190 
.... 60/39.33 

395/22 

395/23 

395/22 

.. 73/40.5 A 
364/474.17 

395/22 

395/23 

395/751 



OTHER PUBLICATIONS 



Widrow et al.. Neural Nets for adaptive filtering and adap- 
tive pattern recognition. Computer, IEEE 1988. pp. 25-39, 
Mar. 1988. 

Broomhead et al.. A parallel architecture for nonlinear 
adaptive filtering and pattern recognition. IEEE conf on 
artificial Neural network. 1989, pp. 265-269. 



Hill et al.. Antenna beamforming for EW using adaptive 
layered networks, IEE Colloquium on 'Signal processing in 
Electronic Warfare* (Digest #1994/025). pp. 2/1-5. Jan. 31. 
1994. 

Dony et al.. Optimally integrated adaptive learning. 1993 
ICASSP, pp. 609-12. Apr. 30. 1993. 

Blacknell et al.. Optimum classification of non-Gaussian 
processes using neural networks. IEEE proceedings, pp. 
55-66. Feb. 1994. 

Ko et al.. Neural network based novelty filtering for signal 
detection enhancement. 1992 IEEE Midwest Symposium, 
pp. 252-255. 

Lippman (1987) "An Introduction to Computing with 
Nueral Nets". 



Primary Examiner — Robert W. Downs 
Assistant Examiner — Sanjiv Shah 

Attorney, Agent, or Firm— Terry J. Anderson; Karl J. Hoch. 
Jr. 



[57] 



ABSTRACT 



An adaptive filtering neural network classifier for classifying 
input signals, includes a neural network and one or more 
adaptive filters for receiving input analog signals to be 
classified and generates inputs for the classifier. Each adap- 
tive filter is characterized as having a predetermined number 
of operating parameters. An analog to digital converter 
converts each input signal into a digital signal before input 
to the neural network. The neural network processes each 
digital signal to generate therefrom a plurality of weighted 
output signals in accordance with the type of network 
implemented. One of the weighted output signals represents 
a class for the input signal, and an error signal representing 
a difference between the weighted output signals and a 
predetermined desired output is also generated by the net- 
work. A control device responsive to the error signal gen- 
erates a further set of operating filter parameters for input to 
each of the adaptive filters to change the operating response 
thereof and mini mi ye the error signal. 
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ABSTRACT 



Emerging patterns (EPs) are itemsets having supports that 
change significantly from one dataset to another. A classifier, 
CAEP, is disclosed using the following main ideas based on 
EPs: (i) Each EP can sharply differentiate the class mem- 
bership of a (possibly small) fraction of instances containing 
the EP, due to the big difference between the EP's supports 
in the opposing classes; the differentiating power of the EP 
is defined in terms of the EP's supports and ratio, on 
instances containing the EP. (ii) For each instance t, by 
aggregating (124) the differentiating power of a fixed, auto- 
matically selected set of EPs, a score is obtained for each 
class (126). The scores for all classes are normalized (144) 
and the largest score determines t's class (146). CAEP is 
suitable for many applications, even those with large vol- 
umes of high dimensional data. CAEP does not depend on 
dimension reduction on data and is usually equally accurate 
on all classes even if their populations are unbalanced. 
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ABSTRACT 



A divide-and-conquer (DAQ method and system improve 
the detection of abnormalities, like lung nodules, in radio- 
logical images via the use of zone-based digital image 
processing and artificial neural networks. The DAC method 
and system divide the lung zone into different zones in order 
to enhance the efficiency in detection. Different image 
enhancement techniques are used for each different zone to 
enhance nodule images, as are different zone-specific tech- 
niques for selecting suspected abnormalities, extracting 
image features corresponding to selected abnormalities, and 
classifying the abnormalities as either true or false abnor- 
malities. 

102 Claims, 11 Drawing Sheets 
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ABSTRACT 



An automatic text classification system is provided which 
extracts words and word sequences from a text or texts to be 
analyzed. The extracted words and word sequences are 
compared with training data comprising words and word 
sequences together with a measure of probability with 
respect to the plurality of qualities. Each plurality of quali- 
ties may be represented by an axis whose two end points 
correspond to mutually exclusive characteristics. Based on 
the comparison, the texts to be analyzed are then classified 
in terms of the plurality of qualities. In addition, a fuzzy 
logic retrieval system and a system for generating the 
training data are provided. 
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US -CL- CURRENT: 715/500 
ABSTRACT : 

An automatic text classification system is provided which extracts words and 
word sequences from a text or texts to be analyzed. The extracted words and 
word sequences are compared with training data comprising words and word 
sequences together with a measure of probability with respect to the plurality 
of qualities. Each plurality of qualities may be represented by an axis whose 
two end points correspond to mutually exclusive characteristics. Based on the 
comparison, the texts to be analyzed are then classified in terms of the 
plurality of qualities. In addition, a fuzzy logic retrieval system and a 
system for generating the training data are provided. 



Summary of Invention Paragraph - BSTX (9) : 

[0009] U.S. Pat. No. 5,781,879 discloses a system for the semantic 
analysis and modification of information in the form of text. A predetermined 
lexicon has scores for lexical units (words or phrases) for various categories. 
Each lexical unit has meaning and semantic content of it's own. The lexicon is 
used to lookup and accumulate an aggregate score for text for each category. A 
user is able to modify the text to modify the semantic content of the text by 
referring the aggregate scores and trying to modify them to preferred values by 
replacing lexical units in the text with lexical units having different scores 
for the categories. This system requires a predetermined lexicon having 
predetermined scores for lexical units for the categories. Each category is 
given a discrete score and a score is assigned for each category only for 
individual lexical units. Thus the accumulated score is accumulated using only 
discrete values for single lexical units and does not provide a system that 
uses rich semantic information in the text and in training texts. 

Detail Description Paragraph - DETX (125) : 

[0162] FIG. 13 illustrates the hierarchical structure of a classification 
tree in accordance with an embodiment of the present invention. In this 
embodiment the qualities or axes have extreme values indicating how much the 
document is concerned with a topic such as Money. Thus the extremes can be 
simply YES and NO. This hierarchical structure requires 4 classifiers having 4 
different sets of training data. In this embodiment the documents are all from 
the Reuters news feed. A first set of training data and a first classifier 
will thus provide 3 qualities or axes for which the documents are given scores 
by automatic or manual classification. The word stems and word stem sequences 
in the documents are identified to obtain the training data which will give 
scores for the 3 axes: Grain, Money and Crude and the associated distribution 
of word stem and word stem sequence scores across the groups as described above 
and as illustrated in FIG. 7. A second set of training data and a second 
classifier will provide 2 qualities or axes: Corn and Wheat for which a subset 
of the documents having the highest scores for the Grain classification are 
given scores by automatic or manual classification. The word stems and word 
stem sequences in the subset of documents are identified to obtain the training 
data which will give scores for the 2 axes: Corn and Wheat and the associated 
distribution of word stem and word stem sequence scores across the groups as 
described above and as illustrated in FIG. 7. A third set of training data and 
a third classifier will provide 2 qualities or axes: Dollar and Interest for 
which a subset of the documents having the highest scores for the Money 
classification are given scores by automatic or manual classification. The 
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word stems and word stem sequences in the subset of documents are identified to 
obtain the training data which will give scores for the 2 axes: Dollar and 
Interest and the associate distribution of word stem and word stem sequence 
scores across the groups as described above and as illustrated in FIG. 7. A 
fourth set of training data and a fourth classifier will provide 2 qualities or 
axes: Gas and Shipping for which a subset of the documents having the highest 
scores for the Crude classification are given scores by automatic or manual 
classification. The word stems and word stem sequences in the subset of 
documents are identified to obtain the training data which will give scores for 
the 2 axes: Gas and Shipping and the associated distribution of word stem and 
word stem sequence scores across the groups as described above and as 
illustrated in FIG. 7. Thus the highest score for one of the qualities or axes 
will determine the classification assigned e.g. Money and hence the next set of 
classifications e.g. Dollar and Interest. 
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